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Executive Summary 


Ensuring that students are ready for college when they graduate from high school has 
important implications for students, educators, education policymakers, and other 
stakeholders. Thus, the research literature has identified several indicators for college 
preparedness, including school attendance, GPA, test scores, and college entrance exams. This 
study focuses on an examination of the relationship between the National Assessment of 
Educational Progress (NAEP) grade 12 mathematics assessment and college entry to identify 
benchmarks on the NAEP grade 12 Mathematics Assessment that are indicative of the skills 
prerequisite for college enrollment. 


NAEP has been the gold standard for assessing what the nation’s students know and can do at 
grades 4, 8, and 12 in reading, mathematics, and science as well as other subject areas. At each 
grade level and for each subject, cut points on the NAEP scale have been established to indicate 
whether performance is at the NAEP Advanced, NAEP Proficient, or NAEP Basic level or whether 
it is below the NAEP Basic level. Thus, it is logical to suppose that grade 12 NAEP performance 
should be indicative of whether students are prepared for college. In fact, studies 
commissioned by the National Assessment Governing Board (NAGB) indicate that NAEP can be 
used to predict college preparedness (Fields, 2014). However, one of the key studies used data 
from only one state, Florida, to establish an indicator of college preparedness: a score of 163 on 
the grade 12 NAEP mathematics assessment. A recent study by Ogut, Bohrnstedt, & Broer 
(2019), using a subsample of students from the High School Longitudinal Study of 2009 
(HSLS:09) base-year collection who also took the 2013 NAEP grade 12 Mathematics Assessment 
(i.e., “the 2013 NAEP overlap sample”), showed that college admission benchmarks can be 
established on the NAEP assessment. 


The purpose of the current study is to extend the analyses from the prediction of college 
admission to actual college enrollment as well as to additional indicators, such as persistence in 
college and the selectivity of the college attended. Using data from HSLS:09 and the 2013 NAEP 
overlap sample, this study examines the relationship between NAEP achievement and college 
enrollment as well as other college-related outcomes. The first set of analyses uses grade 12 
NAEP mathematics achievement to model the probability of enrollment in postsecondary 
education with or without remediation, by the selectivity of the colleges enrolled in, by 
persistence in postsecondary education, and by majoring in a STEM field. The second set of 
analyses examines the ability of grade 12 NAEP mathematics achievement compared to 
achievement on the SAT mathematics college entrance exam to predict postsecondary 
outcomes (e.g., enrollment in a 4-year college), with and without controlling for high school 
GPA. 


Key Findings 


Overall, the results showed that NAEP mathematics achievement explained a considerable 
amount of the variation in postsecondary outcomes: about 28% in overall postsecondary 
enrollment and 34% in the selectivity of the colleges enrolled in. However, the variance 
explained decreased considerably in predicting enrollment without remediation, choosing a 
STEM major, and persistence: 23%, 17%, and 6%, respectively. 


When translated into probabilities, the performance at the cut score for the NAEP Basic 
achievement level corresponded to a 33% probability of entry into a 4-year college: 12% into a 
4-year that was neither classified nor inclusive, 16% into moderately selective one, and a 5% 
into a highly selective one. A student at the NAEP Proficient cut score had a 64% probability of 
entry into a 4-year college: 15% into a 4-year that was neither classified nor inclusive, 33% into 
moderately selective one, and 18% into highly selective one. 


Performance at the NAEP Basic achievement level was also associated with a 49% probability of 
enrolling in postsecondary education without remedial coursetaking, a 13% probability of 
choosing a STEM major, and a 72% probability of persisting in college at least until February 
2016 (the junior year in college for those who attended each year after high school). 
Performance at the NAEP Proficient level, instead, corresponded to a 77% probability of 
enrollment without remedial coursetaking, a 28% probability of choosing a STEM major, and an 
84% probability of persistence at least until February 2016. 


NAEP’s college preparedness indicator was associated with a 53% probability of enrolling in a 
4-year college: 15% a 4-year that was neither classified nor inclusive, 27% into moderately 
selective one, and 11% into a highly selective one. At the NAEP college preparedness indicator 
point, the probability of enrollment without remedial coursetaking was 68%; of choosing a 
STEM major, 21%; and of persisting in college at least until February 2016, 80%. 


There were differences in the relationships between NAEP and the various outcomes by 
subgroup. Results showed that at a given NAEP achievement level, Black students had higher 
probabilities of postsecondary enrollment than White or Hispanic students. Both Black and 
White students had higher probabilities than Hispanic students of enrollment in more selective 
colleges, as well as higher probabilities of choosing STEM majors. White and Black students also 
had higher probabilities of postsecondary enrollment without remediation than did Hispanic 
students. The study also found that at each achievement level, female students had higher 
probabilities of postsecondary enrollment, enrolling in colleges with higher selectivity, enrolling 
without the need for remedial coursetaking, and persisting in college than male students. 
However, male students had higher probabilities than females of choosing STEM majors in 
4-year colleges. 


Compared with the SAT mathematics college admissions test, the NAEP grade 12 Mathematics 
Assessment was about equally good at predicting students’ enrollment into postsecondary 
education, the selectivity of the colleges enrolled in, remedial coursetaking, choosing a STEM 
major, and persistence when controlling for overall high school GPA. 
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Background 


There is a growing consensus that earning a high school diploma is not enough to prepare 
students for college. Reports estimate that only about one-third of students in the nation are 
prepared for college when they graduate from high school (ACT, 2017). The rates are not much 
better for those students who show an interest in postsecondary education by taking a college 
admissions test. Among students who took the SAT in 2017, only 46% were found ready to take 
and succeed in entry-level, college credit-bearing courses. Even when students are admitted to 
college, they often are not academically ready and they often take remedial coursework. In the 
2010-11 school year, 29% of students in 4-year public colleges and 41% of students in 2-year 
colleges reported taking remedial courses (Skomsvold, 2014). According to transcript data from 
the Beginning Postsecondary Students (BPS) longitudinal study, remedial coursetaking rates 
among a cohort of students who entered college in the 2003-04 school year were 39% for those 
in public 4-year institutions and 68% for those in public 2-year institutions (Radford and Horn, 
2012). 


The concern over college preparedness has led to several policy initiatives. In 2009, the U.S. 
Department of Education announced the Race to the Top (RTT) grant to encourage and reward 
states for innovation in K-12 education. One of the conditions for funding was that states adopt 
college- and career-ready proficiency standards. In addition, one of the aims of the Common 
Core State Standards, which have been adopted by many states, is to ensure that all students 
graduate from high school ready to succeed in college and in their careers. 


At the high school level, the most frequently noted college preparedness indicators are high 
school attendance, GPA, and college admission test scores. For example, missing no more than 
10 percent of school days per grade level has been shown to be associated with being on-track 
for high school graduation (Allensworth and Easton, 2007). Maintaining a GPA of 3.0 or higher, 
or passing a high school exit exam or college entrance exam, has also been shown to be 
correlated with college entrance and successfully completing credit-bearing, entry-level college 
courses (Mishook et al., 2012). However, these indicators are of limited utility because they are 
not collected by all states. Similarly, the SAT and ACT tests, which are indicative of future 
performance in college, are taken by only a select number of students. Because of these 
limitations, it would be beneficial to both policymakers and the public to have benchmarks 
signifying college preparedness derived from a test with a common metric that could be used 
by all states. Such a test is the National Assessment of Educational Progress (NAEP). 


NAEP has long been viewed as the “gold standard” for assessing what students know and can 
do in grades 4, 8, and 12. Going back to 1990, students across the country have been regularly 


1 See https://reports.collegeboard.org/archive/sat-suite-program-results/2017/class-2017-results. 


assessed in NAEP in key subjects such as reading, mathematics, science, and writing. 
Furthermore, all states are required to participate in the NAEP reading and mathematics 
assessments at grades 4 and 8 if they are to receive Title 1 funds from the federal government. 


Thus, the grade 12 NAEP assessment would seem to be the perfect candidate on which to 
establish college preparedness benchmarks. In fact, for over a decade the National Assessment 
Governing Board (NAGB) has used it to carry out a series of such studies. One study, which used 
longitudinal data from the state of Florida, arrived at a NAEP preparedness benchmark on the 
grade 12 NAEP mathematics assessment and showed that NAEP could be used to predict 
acceptable end-of-first-year college performance, where acceptable performance was defined 
as a 2.67 (B-) or better grade point average. However, the study was limited in that the data 
came from just a single state, and students who attended out-of-state colleges or non-public 
colleges in Florida could not be followed (Fields, 2014). 


To alleviate the limitations of the NAGB Florida study, Ogut, Bohrnstedt, and Broer (2019) used 
data for a subsample of students from the High School Longitudinal Study of 2009 (HSLS:09) 
who had also been selected to participate in the grade 12 NAEP 2013 mathematics assessment. 
(Note that HSLS:09 constitutes a nationally representative sample of 9th-graders in 2009.) The 
study showed that grade 12 NAEP achievement can be used to predict students’ preparedness 
for college admissions and establish college admission benchmarks. However, the data only 
permitted an examination of the relationship of grade 12 NAEP mathematics to college 
admission and enrollment plans because actual enrollment data was not yet available. 


The purpose of the current study is to extend the work on college preparedness benchmarks for 
the NAEP grade 12 Mathematics Assessment to college enrollment and other indicators, using 
data from HSLS:09’s second follow-up in 2016, 3 years after on-time graduation from high 
school. The second follow-up collected data from students on their postsecondary enrollment, 
major chosen, and persistence? in college. This study uses these data to examine the 
relationship between grade 12 NAEP mathematics scores and college enrollment, 
postsecondary persistence, and the need to take remedial courses. The data also allow for the 
identification of benchmarks on NAEP for different subgroups associated with different 
probabilities of college enrollment. 


Research Questions 
The current study examined the following research questions: 


Research Question 1: What is the relationship between performance on the NAEP Grade 12 
Mathematics Assessment and enrollment in postsecondary education, enrollment without 


* Persistence is a composite variable created by HSLS:09 that shows the enrollment status of a student in February 
2016, which corresponds to the second semester of the junior year for those students who enrolled in college 
immediately after high school. 


remedial coursetaking, enrollment in colleges with different degrees of selectivity, choosing a 
STEM major in 4-year colleges, and persistence in postsecondary education? 


e Does this relationship vary by gender, race/ethnicity, and SES? 


e What are the probabilities of students who score at the NAEP Basic, NAEP Proficient, 
and NAEP Advanced levels on the grade 12 mathematics assessment enrolling in 
postsecondary education, enrolling without remedial coursetaking, enrolling in colleges 
with different degrees of selectivity, choosing a STEM major, and persistence in 
postsecondary education? 


Research Question 2: How does the NAEP Grade 12 Mathematics Assessment compare to the 
SAT mathematics test in predicting entry into postsecondary education, entry into 
postsecondary education without remedial coursetaking, choosing a STEM major in 4-year 
colleges, and persistence in postsecondary education? 


Methods 


Data 


The study utilized two datasets: (a) the special overlap sample of about 3,470? students who 
participated in the HSLS:09 base-year collection and who took the 2013 grade 12 NAEP 
Mathematics Assessment; and (b) all HSLS:09 students for whom imputed NAEP scores could be 
computed (Ogut, Bohrnstedt & Broer, 2015). 


The HSLS:09 base-year data collection took place in the 2009-10 school year, with a random 
sample of all fall-term U.S. 9th-graders. The first follow-up of HSLS:09 took place in the spring of 
2012 when most sample members were in the spring semester of the 11th grade. As part of the 
study, students took algebra assessments in grades 9 and 11. 


HSLS:09 also collected transcripts for all students who remained in the sample in 2013, which 
included ACT and/or SAT scores if they took either or both assessments. HSLS:09 used 
concordance tables to transform ACT scores into SAT units, although the current study focuses 
only on SAT math scale scores. In addition, from June 1 to January 31, 2014, all students (or 
their parents) who had participated in the baseline and first follow-up studies were surveyed in 
the 2013 HSLS:09 update to determine college enrollment plans. In the second follow-up data 
collection in 2016, 3 years after high school graduation, information was collected on students’ 
postsecondary enrollment, remedial coursetaking, and persistence in college. Data from the 


3 All the numbers in this paper are rounded to the nearest 10, per Institute of Education Sciences (IES) disclosure 
review risk rules. 


base year, the first follow-up, the 2013 update, the high school transcripts, and the second 
follow-up were used for this study. 


Outcomes Studied 


Five outcomes were studied: 


e College enrollment right after high school (0 = not enrolled, 1 = enrolled in a 2-year 
college, 3 = enrolled in a 4-year college). 


e College enrollment without remediation, as reported by the student? (0 = not enrolled, 
1 = enrolled in less than a 2-year college, 2 = enrolled in a 2- or 4-year college with 
remediation, 3 = enrolled in a 2- or 4-year college without remediation). 


e The selectivity of colleges as defined by the Integrated Postsecondary Education Data 
System (IPEDS), which included the following categories: 4-year highly selective, 4-year 
moderately selective, 4-year inclusive, 4-year not classified, and 2-year or less; the 
lowest category in the model was for students who were not enrolled. 


e Whether the student chose a STEM major. Data on who chose a STEM major comes 
from students’ reports of the major they most seriously considered when first entering 
postsecondary education. HSLS:09 classifies STEM majors using the U.S. Department of 
Education’s SMART grant definition, which differs slightly from that used by the National 
Science Foundation.° 


e Persistence in postsecondary education. “Persistence” is a composite variable created 
by HSLS:09 to measure the enrollment status of a student in February 2016, which 
corresponds to the second semester of the junior year for those students who enrolled 
in college right after high school. 


Analysis 


Twenty NAEP plausible value scale scores were imputed for students in the dataset as 
described in Ogut, Bohrnstedt, and Broer (2015). In addition to the variables used in that 
study,® SAT mathematics scores, GPA, college enrollment, remedial coursetaking, college 


4 In the student questionnaire, students were asked whether they had taken any remedial or developmental 
courses between the time they finished high school and February 2016. 

5 This study uses the HSLS:09 variable X4ENTMJST (“considering a major in a STEM field”) to determine whether a 
student was considering a STEM major. Unlike the National Science Foundation’s definition, the Department of 
Education’s SMART grant definition does not include psychology or the social and behavioral sciences, such as 
political science and economics, among the STEM fields. 

5 The imputation model in Ogut, Bohrnstedt, and Broer (2015) included algebra scores at grades 9 and 11, 
race/ethnicity, sex, parental education, receipt of special education services, and whether the first language 
learned was English. 


selectivity, and STEM major were included as predictors in the imputation model since they are 
not part of the NAEP population structure model and therefore not including them in the 
imputation model would have resulted in biased parameter estimates for those variables in 
subsequent analyses. 


To examine the relationship between grade 12 NAEP mathematics achievement and 
postsecondary outcomes (Research question 1), ordered logistic regression was used where the 
outcomes increase consecutively as a latent variable, as modeled below: 


Vi = Bix + & 
where y; is the unobserved measure of postsecondary outcome and x;is the grade 12 NAEP 
mathematics achievement score. Twenty plausible values created for this study were used as 


the grade 12 NAEP mathematics achievement scores. Each analysis was run 20 times, and the 
results were combined to get the average estimates and associated standard errors. 


Similar models were used for all analyses and outcomes. For college enrollment, the outcome 
variable is a three-alternative model (0 = not enrolled; 1 = 2-year or less institution, 2 = 4-year 
institution)’ that is defined as 


y= fifa-i< yj; <a /=0,1,2,3 
where ao = —© and as = ©, with the analytical model then taking the form: 


et —B1xité: etj-1-B1xite; 


POV =D = Gy eaFoite) 4 ea) 


where x vis grade 12 NAEP mathematics performance. For selectivity of colleges enrolled in and 
enrollment without remediation analyses, ordered logistic regression models were run. For 
binary outcomes (choosing a STEM major and persistence), the ordered logistic regression 
models are essentially a logistic regression model. 


Results from regression models were used to compute the probabilities of, for example, college 
enrollment at different NAEP proficiency cut points as well as at the score of 163 on NAEP 
mathematics set by the National Assessment Governing Board as the criterion for being college 
ready. The Governing Board defines preparedness as “...qualify[ing] for placement into 
entry-level college credit courses that meet general education requirements, without the need 
for remedial coursework in mathematics or reading” (Fields, 2013; Fields, 2014). Hypothetical 


7 The sample size for those entering less than 2-year colleges is small, therefore 2-year colleges and 
less-than-2-year colleges were combined into a less-than 4-year college category. 


benchmarks that correspond to a 50% probability of achieving various postsecondary outcomes 
(e.g., enrollment into 4-year colleges) were also established in grade 12 NAEP mathematics as 
part of the current study. 


To compare the performance of NAEP and the SAT in predicting postsecondary outcomes 
(Research question 2), we obtained estimates from models with NAEP or the SAT as the only 
predictor—as well as from models including GPA and NAEP or the SAT— these models were 
similar to model 3 above. Model fit statistics (e.g., pseudo R* squared) were then used to 
compare the relative performance of NAEP and SAT in the prediction of the postsecondary 
outcomes. 


Results 


Imputing plausible values 


As discussed in the Analysis section, the first step in examining the relationship between NAEP 
achievement and postsecondary outcomes was to impute NAEP scores for the full HSLS:09 
population. Four models that corresponded to the different outcomes examined in the study 
were run (Table 1). The results show that the variance explained in the NAEP achievement 
score ranged from 84.7% to 87.5%. 


Table 1. Modified maximum likelihood regression results 


WETat]¢) (= Model 1 Model 2 WW Koyo (=) 3 Model 4 WW Koyo (=) sy 
-6.766 -2.013 -4.607 -5.530 -3.195 
Constant (5.288) (5.031)*** (5.620)*** (6.497)*** (6.187)*** 
1.641 1.823 1.616 1.699 1.678 
Algebra 11 (ON122) pe (0.104)*** (0.124)*** (O2125)F +2 (0.137)*** 
0.917 1.091 0.909 0.840 0.873 
Algebra 9 (0.086)*** (0.089)*** (0.088)*** (0.093)*** (0.093)*** 
-10.639 -13.066 -10.290 -9.764 -8.062 
Black (3'287)72 (3.556)*** (3.362)** (3:520)7* (3.220)* 
-4,359 -6.926 -4,320 -4.838 -3.291 
Hispanic (2.359) (2.350)** (2.380) (2.575) (2.674) 
2.279 0.684 2.060 3.575 -1.322 
Asian (3.058) (2.960) (3.059) (2.704) (3.129) 
-2.189 -4.413 -2.120 -3.668 -1.202 
Other race (2.871) (3.062) (2.921) (2.963) (2.661) 
7.403 5.158 7.439 7.820 4.457 
Male (57.43) ne % (1.651)** (1.768)*** (1.884)*** (1.705)* 


WETat-]¢) (- 


Parent college 
graduate 


Income higher 
than $55,000 


Enrolled in 
4-year college 


Enrolled in less 
than 4-year 
college 


Highly selective 


Moderately 
selective 


Inclusive 


Selectivity not 
classified 4-year 


Selectivity not 
classified 2-year 


Persistence 


STEM major 


GPA 


Remedial 
coursetaking 


Pseudo R? 


#0 < 001, **p <.01, *p<.05. 


Koy (=) ie | 


0.329 
(1.501) 


0.920 
(1.621) 


5.256 
(2.431) 


1.641 
(2.609) 


89372. 


(1.602) 


0.863 


Model 2 


1,172 
(1.572) 


0.794 
(1.653) 


2.509 


(0.775)** 


0.847 


Model 3 


0.128 
(1.520) 


0.922 
(1.652) 


7.512 
(2.953)* 


5.510 
(2.682)* 


2.613 
(3.268) 


5.222 
(3.347) 


1.729 
(2.627) 


8.173 
(1.610)*** 


0.867 


Model 4 


0.970 
(1.641) 


0.993 
(1.792) 


1.927 
(2.130) 


8.933 
(1.862) *** 


0.866 


Koyo (=) es) 


0.315 
(1.709) 


0.798 
(1.797) 


5.130 
(2.017)* 


8.749 
(1.811)*** 


0.875 


SOURCE: U.S. Department of Education, National Center for Education Statistics, National Assessment of 
Educational Progress (NAEP), High School Longitudinal Study of 2009 (HSLS:09). 


Since some of the outcomes were highly correlated (enrollment and selectivity of the colleges 
enrolled in) and some had different samples (e.g., persistence only applies to students who 
enrolled in postsecondary education), we ran different models for each outcome. Using the 


results from these models, five sets of plausible values were created, as described in Ogut, 


Bohrnstedt, and Broer (2015), and used in the analyses that follow. Plausible values created 


using results from model 1 are used in analyses of postsecondary enrollment; from model 2, in 


analyses of postsecondary enrollment (with or without remedial coursetaking); from model 3, 


in analyses of selectivity of colleges enrolled in; from model 4, in analyses of persistence; and 
from model 5, in analyses of enrollment in STEM majors. 


Descriptive Statistics for the Analytic Sample 


Table 2 displays the characteristics of the analytic sample. White students accounted for the 
majority of the analytic sample (54%). Hispanic students accounted for 21%, and Black and 
Asian students made up 13% and 4% of the sample, respectively. Thirty-seven percent of the 
students had a parent who was a college graduate or had a more advanced degree, and 51% of 
the students came from families with annual incomes higher than $55,000. The average NAEP 
scale score for the analytic sample was 150.62, slightly lower than the average scale score for 
the nation in 2013 (153.46). 


Table 2. Characteristics of the analytic sample 


WETar] (<1 Mean SD Min Max 
White 0.54 0.50 0.00 1.00 
Black 0.13 0.33 0.00 1.00 
Hispanic 0.21 0.41 0.00 1.00 
Asian 0.04 0.20 0.00 1.00 
Other race 0.08 0.28 0.00 1.00 
Male 0.50 0.50 0.00 1.00 
Parent college 
graduate 0.37 0.48 0.00 1.00 
Income higher 
than 55K 0.51 0.50 0.00 1.00 
Algebra, 
grade 11 50.91 9.97 24.96 84.91 
Algebra, grade9 50.82 9.83 24.10 82.19 
GPA 2.70 0.80 0.00 4.00 
NAEP score 
(imputed) 150.62 31.29 65.43 254.12 


NOTE: N = 14,000. A comparison of the analytic sample with the 2013 NAEP operational sample is 
shown in table A-1 in appendix A. 

SOURCE: U.S. Department of Education, National Center for Education Statistics, High School 
Longitudinal Study of 2009 (HSLS:09). 


After finishing high school, 27% of the analytic sample had not enrolled in college (table 3). 
About 3% were enrolled in less than 2-year programs, 28% in 2-year programs, and 43% in 


4-year colleges. Twelve percent were enrolled in highly selective colleges, 20% in moderately 
selective colleges, and about 11% in inclusive or not classified 4-year colleges. Seventeen 
percent of those who were enrolled in any postsecondary institution had chosen a STEM major. 
Among those enrolled in any postsecondary institution, 23% reported taking one or more 
remedial courses. By February 2016, 77% of those who enrolled in college after high school 
were still enrolled or had completed a degree. 


Table 3. Postsecondary outcomes for the analytic sample 


Variable Mean SD 
Variable Mean SD 
Not enrolled 0.27 0.44 
Enrolled in less than 2-year college 0.03 0.16 
Enrolled in 2-year college 0.28 0.45 
Enrolled in 4-year college 0.43 0.49 
Highly selective college 0.12 0.33 
Moderately selective college 0.20 0.40 
Inclusive college 0.06 0.23 
Not classified college 0.05 0.22 
Not classified 2-year college 0.28 0.45 
Not classified less than 2-year 0.03 0.16 
STEM major 0.23 0.42 
Remedial coursetaking 0.23 0.42 


Persistence OR; 0.42 


NOTE: N = 14,000 for all variables except STEM major (N = 10,700), remedial course taken (N = 10,000), and 
persistence (N = 10,800), which are limited to students enrolled in postsecondary education. 

SOURCE: U.S. Department of Education, National Center for Education Statistics, High School Longitudinal Study of 
2009 (HSLS:09). 


When broken down by race/ethnicity and gender, the results show some differences in 
enrollment patterns, major selection, and persistence (Table 4). White, Black, and Asian 
students were more likely to enroll in 4-year colleges, whereas Hispanic students were more 
likely to enroll in 2-year colleges. Male students, White students, and Asian students were more 
likely to have chosen a STEM major than were other subgroups. Black and Hispanic students 
took remedial courses more often than did White and Asian students. Finally, White, Asian, and 
female students were more likely to persist than were their counterparts. 


Table 4. Postsecondary outcomes for the analytic sample, by race/ethnicity and gender 


Not Less than 
Group Enrolled 2 year 2-year 4-year STEM Remedial Persistence 
White 0.24 0.02 0.25 0.49 0.18 0.18 0.81 
Black 0.33 0.03 0.25 0.39 0.12 0.29 0.70 
Hispanic 0.31 0.03 0.37 0.29 0.14 0.34 0.73 
Asian 0.15 0.01 0.27 0:57 0.33 0.14 0.87 
Other race 0.30 0.05 0.27 0.38 0.14 0.24 0.70 
Female 0.22 0.03 0.28 0.47 O12 0.23 0.79 
Male 0.31 0.02 0.27 0.39 0.22 0.22 0.75 


SOURCE: U.S. Department of Education, National Center for Education Statistics, High School Longitudinal Study of 
2009 (HSLS:09). 


Predicting Enrollment in Postsecondary Education 


The ordered logistic regression models (Equation number 3) described in the Analysis section 
were run to examine the relationship between NAEP achievement and three levels of 
postsecondary enrollment: not enrolled, enrolled in a 2-year or less college, and enrolled ina 
4-year college. In addition to NAEP achievement, the models also use other demographic 
variables to test whether the relationships between grade 12 NAEP mathematics and 
postsecondary outcomes hold when they are included. Table 5 displays the results from these 
analyses. In model 1, NAEP achievement by itself explains 28% of the total variance in 
enrollment in postsecondary education. The addition of race/ethnicity, gender and GPA, 
algebra scores, and parental education and income across models 2 through 5 increases the 
variance explained to 42%. In model 5, significant results are observed for all variables in the 
model, except grade 9 algebra and the Asian indicator variable. 


The cutoffs in the ordered logistic regression models represent the estimated cut points on the 
latent variable (¥; in model 1 in the Analysis section) used to differentiate a given level from the 
other levels. For example, in model 1, the probability that a student with a given NAEP score 
was not enrolled in college was computed as 


p(y = 0) = p(z < cutoff 1) 


where z represents the predicted value (0.036*NAEP score) and cutoff 1 is 4.164. In other 
words, if a student’s predicted value is less than 4.164, then that student likely was not enrolled 
in college. The likelihood of not enrolling in college increases as the student’s predicted value 
decreases. Similarly, if a student’s predicted outcome is higher than 5.843, then that student 
was likely enrolled in a 4-year college. Since there are three outcomes in this ordered logistic 
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regression, each student will have the likelihood of each of these three outcomes adding up to 
100% and will be more likely to have the outcome with the highest probability. 


Table 5. Ordered logistic regression model results for predicting postsecondary enrollment 


WETi Ee] ¢) (14 Model 1 Model 2 Model 3 Model 4 Model 5 
NAEP 0.036 0.039 0.023 0.017 0.017 
achievement (0.001)*** (0.001)*** (0.002)*** (0.003)*** (0.003)*** 

1.294 1.289 1.254 
GPA (0.064)*** (0.064)*** (0.062)*** 
Algebra, 0.018 0.015 
grade 11 (0.006)** (0.006)* 
0.008 0.004 
Algebra, grade 9 (0.006) (0.005) 
0.747 0.882 0.841 0.970 
Black (0.104) *** (0.107)*** (0.108)*** (0.111)*** 
-0.019 0.211 0.189 0.380 
Hispanic (0.079) (0.078)** (0.077)* (0:073)222 
-0.126 -0.031 -0.048 -0.022 
Asian (0.199) (0.193) (0.193) (0.209) 
-0.004 0.208 0.194 0.266 
Other race (0.093) (0.097)* (0.097)* (0.099)** 
-0.236 -0.209 -0.235 
Male (0.061)*** (0.061)*** (0.062)*** 
Parent college 0.599 
graduate (0.058)*** 
Income higher 0.332 
than $55K (0.068)*** 
Cutoff 1: 4.164 4.605 5572. 5.962 Beil 
No college (OsiS7)) aes (0.186) *** (OMISA) RE (OIG a) Pas (OMIS2) ae 
Cutoff 2: 5.843 6.306 7.507 7.902 7.771 
2 years or less (0.175)*** (0.206) *** (0.209)*** (0.207)*** (0.208)*** 
Pseudo R? 0.282 0.293 0.398 0.400 0.416 
N 14,000 14,000 14,000 14,000 14,000 


***o <,001, **p<.01, *p<.05. 
SOURCE: U.S. Department of Education, National Center for Education Statistics, High School Longitudinal Study of 
2009 (HSLS:09). 


To examine the correspondence between NAEP achievement levels and postsecondary 
enrollment (as well as other outcomes, as reported later), cut-point scores for NAEP 
achievement levels and estimates from model 1 were used to compute the probabilities of 
postsecondary outcomes associated with NAEP achievement levels. Model 1 estimates the 
relationship between NAEP achievement and enrollment outcome without taking into account 
any student characteristics, ensuring that the probabilities computed are the same for all 
students and not adjusted for their demographic characteristics. Figure 1 shows the 
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probabilities of not enrolling, enrolling in a 2-year or less college, and enrolling in a 4-year 
college at the NAEP Basic, NAEP Proficient, and NAEP Advanced achievement cut scores. Since 
the Governing Board has not established a cut point for students performing below NAEP Basic, 
we computed the average NAEP achievement score for those students and used that to 
compute the corresponding probabilities. Therefore, that number represents the average 
probability of Below NAEP Basic students, whereas the numbers for NAEP Basic, NAEP 
Proficient, and NAEP Advanced represent the probabilities of students at the cut point (i.e., the 
minimal probability for the group). 


Note that the probability of enrolling in a 4-year college increases from 33% at the NAEP Basic 
cut score to 64% at the NAEP Proficient cut score, reaching 88% at the NAEP Advanced level. 
The probability of enrolling in a 2-year or less college is 40% at the NAEP Basic level, dropping 
to 27% and 10% at the NAEP Proficient and NAEP Advanced levels, respectively. It is interesting 
to note that students who perform below NAEP Basic have a 19% probability of enrolling in a 
4-year college and a 37% probability of enrolling in a 2-year or less college. What is less 
surprising is that the probability of not enrolling in a postsecondary education institution was 
highest for students who performed below NAEP Basic (45%) and only 2% for those at the NAEP 
Advanced level. Another way to look at the group of students who performed below NAEP Basic 
is that fully 56% attended either a 2- or 4-year college. The NAEP college preparedness 
indicator, on the other hand, was associated with a 33% probability of enrolling in a 2-year or 
less college and a 53% probability of enrolling in a 4-year college. 
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Figure 1. Probabilities of enrollment by NAEP achievement levels, 
and NAEP college preparedness indicator 
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*Significantly different from the NAEP Basic achievement level. 
SOURCE: U.S. Department of Education, National Center for Education Statistics, High School 
Longitudinal Study of 2009 (HSLS:09). 


It should be noted that these probabilities reflect not only the chances of admission but the 
decision to apply to a 4-year or 2-year college or not to apply at all. For example, for students at 
the NAEP Advanced level, the 88% probability of entering a 4-year college represents the fact 
that most high-performing students choose to apply to a 4-year college rather than to a 2-year 
college, as well as to their high rate of admission to 4-year colleges. In addition, since few 
advanced students choose to go to 2 year-colleges, the 9% probability of admission shown in 
figure 1 is mainly a function of the number of students who choose to go rather than of their 
admission chances. If one is interested, the probability of admission alone can be estimated; for 
example, the probability of admission for Advanced students to a 2-year college is estimated at 
97% (see appendix A for more detail). 


Figure 2 displays the probabilities of entering a 4-year college. It shows the same percentages 
as in figure 1 for the NAEP achievement levels as well as the probability of entering a 4-year 
college based on NAEP’s college preparedness indicator (a score of 163 in mathematics). Notice 
that the indicator of 163 corresponds to a 51% probability of enrolling in a 4-year college. 
Hypothetical benchmarks that would yield probabilities of 50%, 66%, and 75% of entering a 
4-year college correspond to scores of 162, 181, and 193, respectively, on the grade 12 NAEP 
mathematics scale. 
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Figure 2. Predicted relationship between NAEP achievement score and 4-year college 
enrollment 
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SOURCE: U.S. Department of Education, National Center for Education Statistics, High School Longitudinal Study of 
2009 (HSLS:09). 


Enrollment without remediation 


Remedial coursetaking information is reported by the student as part of the student 
background survey. To mimic NAEP’s definition of college preparedness, the data were pooled 
across 4-year and 2-year college enrollment to create an ordered variable with four levels: 


e 0: not enrolled; 

e 1: enrolled in a less than 2-year college; 

e 2: enrolled in a 2- or 4-year college with remedial coursetaking; and 

e 3: enrolled in a 2- or 4-year college without remedial coursetaking. 
The ordered logistic regression models (Equation number 3) described earlier were used to 
examine the relationship between NAEP achievement and enrollment in a postsecondary 
college without remediation. Table 6 displays the results from these analyses. In model 1, NAEP 


by itself explains about 10% of the total variance in enrollment without remediation. The 
addition of race/ethnicity, gender and GPA, algebra scores, and parental education and income 
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across models 2 through 5 increases the variance explained up to 13%. In model 5, even with 


the addition of parental socioeconomic status (SES) variables, the amount of variance explained 


remains at 13% but the estimate for NAEP achievement is no longer statistically significant. 


Instead, GPA and grade 9 algebra appear to explain most of the variance. 


Table 6. Logistic regression model results for predicting enrollment without remediation 


Variables 


NAEP 
achievement 
GPA 


Algebra, 
grade 11 


Algebra, grade 9 


Black 
Hispanic 
Asian 
Other race 
Male 


Parent college 
graduate 
Income higher 
than S55K 
Cutoff 1: No 
college 
Cutoff 2: Less 
than 2-year 
Cutoff 3: 2- or 
4-year with 
remediation 


Cutoff 4: 2-year 


or lower 
without 
remediation 
Pseudo R? 
N 


#Rounds to zero. 


Model 1 


0.036 
(0.001)*** 


4.147 
(0.204) *** 
4.301 
(0.213)*** 
5.132 
(0.216) *** 


4.147 
(0.204) *** 


0.234 
14,640 


***o < 001, **p<.01, *p<.05. 
SOURCE: U.S. Department of Education, National Center for Education Statistics, High School Longitudinal Study of 


2009 (HSLS:09). 


WW Keye (=) 


0.038 
(0.002)*** 


0.583 
(0.091)*** 
0.057 
(0.084) 
0.191 
(0.175) 
-0.014 
(0.085) 


4.526 
(0.218) *** 
4.681 
(0.226) *** 
5.520 
(0.230)*** 


4.526 
(0.218) *** 


0.239 
14,640 


WW Keye (=) 3 


0.027 
(0.002)*** 

1,137 
(0.055) *** 


0.788 
(0.103) *** 
0.332 
(0.099)** 
0.154 
(0.171) 
0.174 
(0.091) 
-0.189 
(0.068) ** 


5.817 
(0.221)*** 


5.998 
(0.229) *** 

6.941 
(0.236)*** 


5.817 
(0.221)*** 


0.341 
14,000 


Model 4 


0.028 
(0.003) *** 
1.142 
(0.058) *** 
-0.003 
(0.008) 

# 

(0.005) 
0.796 
(0.108) *** 
0.337 
(0.102)** 
0.156 
(0.170) 
0.178 
(0.092) 
-0.191 
(0.069)** 


5.786 
(0.221)*** 
5.967 
(0.228) *** 
6.911 
(0.234) *** 


5.786 
(0.221)*** 


0.342 
14,000 


WW Keoye (=) es) 


0.027 
(0.003) *** 
1.106 
(0.056) *** 
-0.004 
(0.008) 
-0.002 
(0.005) 
0.874 
(0.109) *** 
0.458 
(0.104) *** 
0.176 
(0.169) 
0.227 
(0.093)* 
-0.206 
(0.071)** 
0.361 
(0.063) *** 
0.261 
(0.065)*** 
5.659 
(0.218) *** 
5.843 
(0.225) *** 
6.795 
(0.232)*** 


5.659 
(0.218) *** 


0.350 
14,000 
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The results from model 1 were then used to estimate the predicted probabilities associated 
with the NAEP achievement levels and enrollment without remediation. The percentages 
shown in figure 3 show the probabilities of enrolling with or without remediation for the NAEP 
Basic, NAEP Proficient, and NAEP Advanced achievement levels and the NAEP college 
preparedness indicator. For students scoring below NAEP Basic’s cut score, their average 
achievement score was used to compute the corresponding percentages. 


The results show that the probability of enrolling in a 2- or 4-year college without remediation 
increased from 49% at the NAEP Basic level to 94% at the NAEP Advanced level. Somewhat 
surprisingly, the average below NAEP Basic student had a 32 percent probability of enrolling 
without remediation. The probability for those at the NAEP college preparedness benchmark 
was 68%. 


Figure 3. NAEP probability benchmarks for enrollment by remediation status in 
postsecondary education, by NAEP achievement level, and college preparedness indicator 
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*Significantly different from the NAEP Basic achievement level. 

SOURCE: U.S. Department of Education, National Center for Education Statistics, High School Longitudinal Study of 
2009 (HSLS:09). 
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Predicting the Selectivity of the College Enrolled in 


The ordered logistic regression models (Equation number 3) described above were used to 
examine the relationship between grade 12 NAEP mathematics achievement and the selectivity 
of the postsecondary institution enrolled in. Selectivity was measured at six levels: 


1 
2 
= 
4. 
5 
6 


not enrolled; 

selectivity not classified, 2-year college; 
selectivity not classified, 4-year college; 
inclusive 4-year college; 

moderately selective 4-year college; and 


highly selective 4-year college. 


As in the earlier analyses, student and parent demographic variables are used as controls in the 
models. Table 7 displays the results from these analyses. In model 1, NAEP achievement by 
itself explains about 26% of the total variance in college selectivity. The addition of 


race/ethnicity, gender and GPA, algebra scores, and parental education, and family income 
increases the variance explained to 39%. In model 5, all of the variables are statistically 
significant, except grade 9 algebra, Asian, and other race. 


17 


Table 7. Ordered logistic regression model results for predicting selectivity of the college 
enrolled in 


WETi Ee] (14 Model 1 Model 2 Model 3 Model 4 Model 5 
NAEP 0.039 0.040 0.026 0.016 0.016 
achievement (0.001) *** (0.001)*** (0.001)*** (0.002)*** (0.002)*** 

1.326 1.308 1.272 
GPA (0.060)*** (0.059)*** (OL057) Fa 
Algebra, 0.030 0.026 
grade 11 (0.005)*** (0.006)*** 
0.011 0.007 
Algebra, grade 9 (0.005)* (0.005) 
0.618 0.762 0.704 0.845 
Black (0.094) *** (0.097)*** (0.096)*** (0.099)*** 
-0.067 0.167 0.134 0.334 
Hispanic (0.082) (0.081)* (0.078) (OLO7/7) eee 
0.055 0.182 0.148 0.188 
Asian (0.177) (0.175) (0.174) (0.183) 
-0.096 0.101 0.078 0.143 
Other race (0.084) (0.086) (0.086) (0.089) 
-0.204 -0.168 -0.184 
Male (0.053)*** (0.053)** (0.053)*** 
Parent college 0.638 
graduate (0.059)*** 
Income higher 0.333 
than S55K (0.061)*** 
Cutoff 1: No 4.475 4.776 5.923 6.526 6.350 
college (0.144) *** (O72) a5 (OMIA) ee (0.180)*** (OMS 1) Ree 
Cutoff 2: 2-year 6.161 6.479 7.873 8.484 8.350 
or lower (0.161)*** (0.190)*** (0.191)*** (0.196)*** (0.196)*** 
Cutoff 3: 
Not classified or 6.446 6.769 8.201 8.814 8.689 
lower (O62) 222 (O:191) 2" (Ox192) "4% (OAS 7) 2 S2 (OAS 7)ES 
Cutoff 4: 
Inclusive or 6.789 7.117 8.591 9.207 9.094 
lower (0.163)*** (0.192)*** (0.194)*** (0.196)*** (0.196)*** 
Cutoff 5: 
Moderate or 8.364 8.705 10.290 10.925 10.864 
lower (Osea) ees (O:201) 22% (OL207)55% (O2O7) Ea (02208) 2% 
Pseudo R? 0.343 0.355 0.446 0.452 0.473 
N 14,000 14,000 14,000 14,000 14,000 


***o < 001, **p <.01, *p<.05. 
SOURCE: U.S. Department of Education, National Center for Education Statistics, High School Longitudinal Study of 
2009 (HSLS:09). 


To determine the correspondence between the NAEP achievement levels and college 
selectivity, the results from model 1 were used to generate predicted enrollment probabilities. 
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In figure 4, the percentages show the probabilities of enrolling in colleges with different 
degrees of selectivity for the NAEP Basic, NAEP Proficient, and NAEP Advanced achievement 
levels and the NAEP college preparedness indicator. For students who fell below the NAEP Basic 
cut score, we again used their average achievement score to compute the corresponding 
percentages. 


The results show that the probability of students enrolling in a highly selective college increased 
from 5% at the NAEP Basic level to 50% at the NAEP Advanced level, whereas their probability 
of enrolling in a moderately selective college was 33% at both the NAEP Proficient and NAEP 
Advanced levels. Students had a 6% probability of enrolling in an inclusive college at the NAEP 
Basic level and a 7% probability at the NAEP Proficient level. Interestingly, below NAEP Basic 
students had about a 10 percent probability of enrolling in a moderately or highly selective 
college. At NAEP’s college preparedness indicator, the probabilities of enrolling in a 4-year 
inclusive college, a moderately selective college, or a highly selective college were 8%, 28%, and 
12%, respectively. Although it is not shown in the figure, what is perhaps most surprising is that 
at the NAEP college preparedness cut point, the probability of either not attending college or 
attending no more than a 2-year college was fairly high, at 45% (not shown in the figure). 


Figure 4. Probabilities of the selectivity of 4-year college enrolled in, by 
NAEP achievement level and NAEP college preparedness indicator 
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*Significantly different than the NAEP Basic achievement level. 
SOURCE: U.S. Department of Education, National Center for Education Statistics, High School Longitudinal Study of 
2009 (HSLS:09). 
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Predicting Persistence in Postsecondary Education 


Information on postsecondary persistence comes from an HSLS:09-derived variable that 
indicates whether a student who enrolled in postsecondary education after high school was still 
enrolled in or had attained a credential at any institution at the time of the second HSLS:09 data 
collection in February 2016. After the data were pooled across enrollment in 4-year and 2-year 
or less postsecondary institutions, logistic regression models were used to examine the 
relationships between NAEP achievement levels and persistence. 


Table 8 displays the results from these analyses. In model 1, NAEP achievement by itself 
explains about 6% of the total variance in persistence. Although the addition of race/ethnicity, 
gender and GPA, algebra scores, and parental education and income in models 2 through 5 
increases the variance explained to 12%, none of the estimates for race/ethnicity, gender, and 
algebra scores are Statistically significant. In model 5, the estimate for NAEP achievement is 
statistically significant, along with GPA and parental education and income. 
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Table 8. Logistic regression model results for predicting persistence in postsecondary 


education 

ETE] (214 Model 1 Model 2 Model 3 Model 4 Model 5 
NAEP 0.020 0.019 0.009 0.010 0.009 
achievement (0.002)*** (0.002)*** (0.002)*** (0.004)* (0.004)* 
GPA 0.883 0.890 0.855 
(0.087)*** (0.084)*** (0.085)*** 
Algebra, 0.002 # 
grade 11 (0.011) (0.012) 
Algebra, grade 9 -0.008 -0.011 
(0.007) (0.007) 
Black 0.009 0.096 0.096 0.210 
(0.175) (0.172) (0.183) (0.176) 
Hispanic -0.079 0.081 0.080 0.242 
(0.166) (0.173) (0.178) (0.172) 
Asian 0.226 0.332 0.338 0.379 
(0.199) (0.204) (0.203) (0.213) 
Other race -0.312 -0.174 -0.172 -0.110 
(0.140)* (0.148) (0.142) (0.151) 
Male -0.127 -0.132 -0.156 
(0.078) (0.081) (0.081) 
Parent college 0.420 
graduate (0.085)*** 
Income higher 0.304 
than S55K (0.082)*** 
Cut 1 1.840 1.741 2.608 2.504 Desi/all 
(OR245) eee (ORZ6H)) ee (O27 A) ees (O2289)2% (08292) 2% 
Pseudo R? 0.064 0.066 0.11 0.11 0.119 
N 10,800 10,800 10,800 10,800 10,800 


#Rounds to zero. 

***o < 001, **p<.01, *p<.05. 

SOURCE: U.S. Department of Education, National Center for Education Statistics, High School Longitudinal Study of 
2009 (HSLS:09). 


The results from model 1 were used to compute the predicted probabilities of postsecondary 
persistence associated with the NAEP achievement levels. The results in figure 5 show the 
probabilities of persistence increasing from 72% at the NAEP Basic level to 92% at the NAEP 
Advanced level. Not surprisingly, the probability of dropping out is highest for below NAEP Basic 
students. That noted, it is perhaps surprising that 63% of below NAEP Basic students who 
started college after high school had either completed or were still enrolled in college 3 years 
later. As expected, the probability of persistence for those scoring at NAEP’s college 
preparedness benchmark lies between the probabilities associated with the NAEP Basic and 
NAEP Proficient levels, at 80%. 
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Figure 5. NAEP probability benchmarks for persistence in postsecondary education, by 
NAEP achievement level and NAEP college preparedness indicator 
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*Significantly different from the NAEP Basic achievement level. 
SOURCE: U.S. Department of Education, National Center for Education Statistics, High School Longitudinal 
Study of 2009 (HSLS:09). 


Predicting the Choice of a STEM major 


The study also examined the relationship between NAEP achievement and enrolling in a STEM 
major, based on students’ reports of the major they most seriously considered when entering 
postsecondary education. HSLS:09 classifies STEM majors using the U.S. Department of 
Education’s SMART grant definition, which differs slightly from that used by the National 
Science Foundation.® 


For this set of analyses, the sample was limited to those who entered 4-year colleges (see 
table 9). In model 1, NAEP achievement by itself explains about 17% of the total variance in 
choosing a STEM major. Moving from models 2 through 5, the addition of race/ethnicity, GPA 
and gender, algebra scores, and parental education and income increases the variance 
explained to 19.5%. In model 5, NAEP achievement and GPA remain statistically significant in 
the presence of these control variables. 


8 This study uses the HSLS:09 variable X4ENTMJST (“considering a major in a STEM field”) to determine whether a 
student was considering a STEM major. Unlike the National Science Foundation’s definition, the Department of 
Education’s SMART grant definition does not include psychology or the social and behavioral sciences, such as 
political science and economics, among the STEM fields. 
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Table 9. Logistic regression model results for predicting the choice of a STEM major in 
postsecondary education 


Welt] ¢) (3 Model 1 Model 2 Model 3 Model 4 Model 5 
NAEP 0.038 0.039 0.029 0.021 0.021 
achievement (0.002)*** (0.002)*** (0.003)*** (0.004)*** (0.004)*** 
GPA 0.818 0.759 0.741 

(0.089)*** (0.086)*** (0.086) *** 

Algebra, 0.023 0.021 

grade 11 (0.010 )* (0.010 )* 

Algebra, grade 9 0.012 0.012 

(0.007) (0.007) 

Black 0.587 0.728 0.711 0.744 

(0.179)** (0.172)*** {0,1.71)*** (0.167)*** 

Hispanic -0.044 0.115 0.094 0.149 

(0.168) (0.187) (0.185) (0.177) 

Asian 0.431 0.495 0.424 0.426 

(0.198)* (0.192)* (0.197)* (0.195)* 

Other race 0.135 0.256 0.248 O:257; 

(0.162) (0.165) (0.165) (0.164) 

Male 0.770 0.769 0.781 

(0.090)*** (0.091)*** (0.089)*** 

Parent college 0.212 

graduate (0.117) 

Income higher 0.013 

than S55K (0.106) 

Cutoff 1: Not 7.876 8.142 9.416 9.899 9.818 

persisting in (0305) 2% (OS 7a) ees (0.389)*** (0.405)*** (0.418)*** 
postsecondary 

Pseudo R? 0.171 0.176 0.190 0.193 0.195 

N 10,700 10,700 10,700 10,700 10,700 


FRED < 001, ** p< 01, *p:<:.05. 
SOURCE: U.S. Department of Education, National Center for Education Statistics, High School Longitudinal Study of 
2009 (HSLS:09). 


The results from model 1 were used to estimate the predicted probabilities of choosing a STEM 
major associated with the NAEP achievement levels. Figure 6 shows that the probabilities 
increase from 13% at the NAEP Basic level to 52% at the NAEP Advanced level. That is, over half 
of the students at the NAEP Advanced level are predicted to choose a STEM major, compared to 
8% of the NAEP Basic students. Finally, the predicted probability that a student scoring at 
NAEP’s college preparedness benchmark will choose a STEM major is 21%, or 7% lower than the 
probability associated with scoring at the NAEP Proficient level. 


Zs 


Figure 6. NAEP probability benchmarks for choosing a STEM major in 4-year colleges, 
by NAEP achievement level and NAEP college preparedness indicator 
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*Significantly different from the NAEP Basic achievement level. 
SOURCE: U.S. Department of Education, National Center for Education Statistics, High School Longitudinal 
Study of 2009 (HSLS:09). 


Subgroup Analyses 


The previous results used data from all students to examine the relationship between NAEP 
achievement and postsecondary outcomes. The following set of results are subgroup analyses 
derived from the same models described previously but using only data from the population of 
interest. For example, in the case of racial/ethnic subgroups, model 3 was run separately for 
each racial/ethnic group reported. As before, the results from the subgroup analyses are 
grouped by the outcome of interest. 


Predicting Enrollment in Postsecondary Education 


After estimating ordered logistic regression models where NAEP achievement is used to predict 
postsecondary enrollment, the results were used to compute a model to predict the 
probabilities of college enrollment by gender, race/ethnicity, and parental education. As shown 
in figure 7, there were some differences across student groups. Compared to male students, 
female students had higher probabilities of enrolling in 4-year colleges at every NAEP 
achievement level, including below NAEP Basic. Male students were also more likely not to 
enroll in any postsecondary education. Across racial/ethnic groups, at each achievement level, 
Black students had the highest probabilities of enrolling in 4-year colleges, followed by White 
students, then Hispanic students. At the NAEP Basic level, Hispanic students had a 50% chance 
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of enrolling in a 2-year college or less, compared with 34% for Black students and 37% for 
White students. At each achievement level, students whose parents graduated from college or 
had an advanced degree were more likely to enter 4-year colleges than were those students 
whose parents had less than a college education. 
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Figure 7. Probabilities of college enrollment, by NAEP achievement level, gender, race/ethnicity, and parental education 
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*Significantly different from the reference group (male, White, or parental education less than college). 
SOURCE: U.S. Department of Education, National Center for Education Statistics, High School Longitudinal Study of 2009 (HSLS:09). 
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Predicting Enrollment without Remediation 


Figure 8 displays the probabilities of enrollment with or without remediation generated from 
separate ordered logistic regression models for the subgroups of interest. Across all subgroups, 
the probability of enrolling without remedial courses upon enrollment increased when moving 
from the NAEP Basic to the NAEP Advanced level. Similar to the results for overall enrollment, 
where female students higher probabilities of enrolling in 4-year colleges, they had higher 
probabilities of enrolling without remedial courses than did male students at each NAEP 
achievement level. 


In addition to higher probabilities of enrolling in 4-year colleges, Black students had higher 
probabilities of enrolling without remedial coursetaking than did White students at all NAEP 
achievement levels, except Advanced. Hispanic students, on the other hand, had lower 
probabilities of enrolling without remedial coursetaking across most achievement levels. At 
each NAEP achievement level, students whose parents graduated from college or had an 
advanced degree had higher probabilities of enrolling without remedial coursetaking than did 
students whose parents had less than a college education. 
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Figure 8. Probabilities of college enrollment without remediation, by NAEP achievement level, gender, race/ethnicity, and 


parental education 
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*Significantly different from the reference group (male, White, or parental education less than college). 
SOURCE: U.S. Department of Education, National Center for Education Statistics, High School Longitudinal Study of 2009 (HSLS:09). 
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Predicting Selectivity of College Enrolled in 


Figure 9 displays the probabilities of enrolling in colleges with different degrees of selectivity 
based on analyses from ordered logistic regression models for the subgroups of interest. The 
results are from models that included all degrees of selectivity, from not enrolled to highly 
selective colleges, although the figure displays the results only for 4-year colleges. First looking 
at differences by gender, female students had higher probabilities of enrolling in a highly 
selective college at each NAEP achievement level, which is consistent with the finding reported 
earlier for overall enrollment. At the NAEP Proficient level, female students also had a higher 
probability than male students of enrolling in a moderately selective college. In contrast, male 
students were equally likely to enroll in either a moderately selective college or a less than 
4-year college. 


Across racial/ethnic groups, Black students had a similar probability as White students of 
enrolling in highly selective colleges at each achievement level, whereas Hispanic students had 
a lower probability than both Black students and White students. Black students at the NAEP 
Proficient level had a 49% probability of enrolling in a moderately selective 4-year college, 
compared with a 34% probability for White students and a 27% probability for Hispanic 
students. Hispanic students at the NAEP Proficient level had a 37% probability of enrolling in a 
less-than 4-year college. 
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Figure 9. Probabilities of enrolling in 4-year colleges with different degrees of selectivity, by NAEP achievement level, gender, 


race/ethnicity, and parental education 
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*Significantly different from the reference group (male, White, or parental education less than college). 
SOURCE: U.S. Department of Education, National Center for Education Statistics, High School Longitudinal Study of 2009 (HSLS:09). 


Predicting Persistence in Postsecondary Education 


In terms of postsecondary persistence, female students had higher probabilities than male 
students of persisting in college at each NAEP achievement level, whereas Black students had 
lower probabilities than either White or Hispanic students, although the differences were not 
statistically significant (figure 10). At each NAEP achievement level, the probabilities of 
persistence were higher for students whose parents graduated from college or had advanced 
degrees than for students whose parents had less than a college education. 
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Figure 10. Probabilities of postsecondary persistence in 4-year colleges, by NAEP achievement level, gender, race/ethnicity, and 


parental education 
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*Significantly different from the reference group (male, White, or parental education less than college). 
SOURCE: U.S. Department of Education, National Center for Education Statistics, High School Longitudinal Study of 2009 (HSLS:09). 
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Predicting the Choice of a STEM Major 


Figure 11 shows the probabilities of choosing a STEM major in 4-year colleges for all subgroups 
of interest by NAEP achievement level. Although female students at each NAEP achievement 
level had higher probabilities of enrolling in 4-year colleges and colleges with higher selectivity 
than male students, as well as lower probabilities of remedial coursetaking, male students had 
higher probabilities of choosing a STEM major. In addition, at NAEP’s Basic and Proficient levels, 
Black students had higher probabilities than White students of choosing a STEM major. 
However, there were no differences in the probability of choosing a STEM major between 
students whose parents graduated from college or had advanced degrees and students whose 
parents had less than college education; this was true at each NAEP achievement level. 
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Figure 11. Probabilities of choosing a STEM major in 4-year colleges, by NAEP achievement level, gender, race/ethnicity, and 
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*Significantly different from the reference group (male, White, or parental education less than college). 
SOURCE: U.S. Department of Education, National Center for Education Statistics, High School Longitudinal Study of 2009 (HSLS:09). 
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Comparing NAEP and the SAT in predicting postsecondary outcomes 


When colleges make admission decisions, they often use either the SAT or ACT as a measure of 
achievement, often in combination with overall high school GPA. The set of results displayed 
below is designed to examine how the NAEP grade 12 Mathematics Assessment compares to 
the SAT mathematics assessment in the prediction of college enrollment (including the 
selectivity of the college or university chosen), persistence, remediation, and choosing a STEM 
major. For these analyses, ordered logistic regression models as described earlier were used, 
and analyses were limited to the group of students who had SAT scores to ensure the 
comparability of samples across the SAT and NAEP analyses. 


Table A-2 in appendix A compares the two samples. Not surprisingly, students who had SAT 
scores had higher algebra achievement scores (in grades 9 and 11) and higher GPAs than those 
in the analytic sample. The SAT analytic sample also appeared to have more male students and 
higher family SES, but there were no apparent differences in terms of race/ethnicity. The 
zero-order correlations among the SAT, GPA, and NAEP scale scores are shown in table A-3 in 
appendix A. 


Predicting Enrollment in Postsecondary Education 


Table 10 displays the results of the analyses for predicting postsecondary enrollment. In model 
|, using NAEP scores as the only predictor explains about 20% of the variation in postsecondary 
enrollment, which is almost identical to using SAT mathematics scores as the only predictor 
(model 2). When GPA is added as a covariate, the variance explained increased to about 28% 
for both the NAEP and the SAT models (model 3 vs. model 4). That is, the NAEP grade 12 
Mathematics Assessment did as well in explaining the variance in postsecondary enrollment as 
did SAT mathematics when examined in combination with high school GPA. 
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Table 10. Comparing NAEP and SAT mathematics scores in predicting postsecondary 


enrollment 
WETar) 9) (=) WY Coye (=) fmf 
0.033 
NAEP (0.001)*** 
GPA 
SAT 
3.048 


Cutoff 1: Nocollege §(0.195)*** 
Cutoff 2: 2 years or 4.816 


less (0.213)*** 
Pseudo R? 0.204 
N 10,660 


***D < 001, **p <.01, *p<.05. 


Koy (=) pd 


0.010 
(0.000)*** 
2.783 
(0.177) *** 
4.541 
(0.203) *** 
0.198 
10,660 


Koyo f=) fae} 


0.020 
(0.002) *** 
1.167 

(0.074) *** 


4.218 
(0.197) *** 
6.127 
(0.207) *** 
0.276 
10,660 


WW Kore f=) 2} 


1.189 
(0.064) *** 
0.006 
(0.000) *** 
4.292 
(0.195) *** 
6.210 
(0.213) *** 
0.280 
10,660 


SOURCE: U.S. Department of Education, National Center for Education Statistics, High School Longitudinal Study of 


2009 (HSLS:09). 


Predicting Selectivity of Enrolled Colleges 


The next set of analyses compared NAEP and the SAT in terms of predicting the selectivity of 
the college enrolled in (table 11). Comparing models 1 and 2, the results indicate that NAEP 
explained about 28% of the variance in selectivity, whereas the SAT explained about 30%. 


When GPA is added to the models, the model that includes NAEP explained 36% of the 


variance, compared to 39% for the model including SAT and GPA as the predictor variables 


(model 4). That is, performance on the NAEP grade 12 Mathematics Assessment explained only 


slightly less variance than SAT mathematics in the prediction of the selectivity of the college 


enrolled in. 
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Table 11. Comparing NAEP and SAT mathematics scores in predicting selectivity of college 


enrolled in 
Variable Model 1 WV CoXe [=] fd WY oXe [=] Model 4 
0.036 0.024 
NAEP (0.001)*** (0.002)*** 
1.188 77, 
GPA (0.068)*** (0.056)*** 
0.012 0.008 
SAT (0.000)*** (0.000)*** 
3517 3.312 4.834 4.931 
Cutoff 1: Nocollege (0.181)*** (0.148)*** (O:192)*** (O:176)*** 
Cutoff 2: 2-year or 5.294 5.089 6.762 6.872 
lower (0.190)*** (0.149)*** (0.193)*** (O:175)*** 
Cutoff 3: Not 5.607 5.411 7.103 e222) 
classified or lower (0.194)*** (OMIS1) ee (0.198)*** (Ol179)*=* 
Cutoff 4: Inclusive 6.005 5.822 7.533 7.666 
or lower (0.196)*** (0.152)*** (0.202)*** (0.181)*** 
Cutoff 5: Moderate 7.740 7.622 9.355 9.557 
or lower (O:210)F4* (On163) Rae (Oia) Rae (0.193)*** 
Pseudo R? 0.285 0.307 0.357 0.381 
N 10,660 10,660 10,660 10,660 


***o < 001, **p <.01, *p<.05. 
SOURCE: U.S. Department of Education, National Center for Education Statistics, High School Longitudinal Study of 
2009 (HSLS:09). 


Predicting Enrollment Without Remediation 


Table 12 displays the results of the analyses comparing NAEP and the SAT in terms of predicting 
postsecondary enrollment without remediation. The results indicate that NAEP explained about 
17% of the variance, whereas the SAT explained about 16%. When GPA is added to the models, 
the model that included NAEP explained 23% of the variance, compared to 21% for the model 
including SAT and GPA as the predictor variables (model 4). These results indicate that SAT and 
NAEP explained a similar amount of variance in predicting postsecondary enrollment without 
remediation when the model included high school GPA. 
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Table 12. Comparing NAEP and SAT mathematics scores in predicting postsecondary 
enrollment without remediation 


Variable Model 1 WY CoXe (=) fd WY oXe (=) Model 4 
0.034 0.023 
NAEP (0.001)*** (0.002)*** 
0.871 0.925 
GPA (0.078)*** (O1072)F** 
0.011 0.007 
SAT (0.000)*** (0.000)*** 
3.116 2.893 3.972 4.062 
Cutoff 1: Nocollege (0.214)*** (O}191) 2+ (0.208)*** (O:232)2** 
Cutoff 2: Less than 3.277 3.050 4.141 4.229 
2-year (0.220)*** (0.180)*** (0.208)*** (0.220)*** 
Cutoff 3: 2- or 
4-year with 4.382 4.146 5.295 5.385 
remediation (O:232)7 42 (0.190)*** (O.224)*** (O235)24* 
Pseudo R? 0.170 0.155 0.216 0.213 
N 11,070 11,070 10,660 10,660 


***o < 001, **p<.01, *p<.05. 
SOURCE: U.S. Department of Education, National Center for Education Statistics, High School Longitudinal Study of 
2009 (HSLS:09). 


Predicting Persistence in Postsecondary Education 


Table 13 displays the results of the analyses comparing NAEP and the SAT in predicting 
persistence in postsecondary enrollment. The results indicate that NAEP explained about 10% 
of the variance in persistence, whereas the SAT explained about 13%. When GPA is added to 
the models, the model that included NAEP explained 12% of the variance, compared to 13% for 
the model including SAT and GPA as the predictor variables (model 4). These results indicate 
that the SAT explained a slightly greater amount of variation by itself than did NAEP. However, 
with the addition of GPA, grade 12 NAEP mathematics explained virtually the same amount of 
variance in persistence as the SAT mathematics score. 
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Table 13. Comparing NAEP and SAT mathematics scores in predicting persistence in 
postsecondary enrollment 


Variable Model 1 WY CoXe (=) fd WY CoXe [=] 3 Model 4 
0.020 0.009 
NAEP (0.002)*** (0.002)*** 
1.018 1.034 
GPA (Opals (0.108)*** 
0.006 0.003 
SAT (0.001)*** (0.001)** 
Cutoff 1: Not 
persisting in 1.703 S75 2901 2.852 
postsecondary (OSI 7 Eee (0.398)*** (0.346)*** (O:385)5=2 
Pseudo R? 0.054 0.050 0.096 0.096 
N 9,360 9,360 9,360 9,360 


***o < 001, **p <.01, *p<.05. 
SOURCE: U.S. Department of Education, National Center for Education Statistics, High School Longitudinal Study of 
2009 (HSLS:09). 


Predicting the Choice of a STEM major 


Table 14 displays the results of the analyses comparing NAEP and the SAT in predicting the 
choice of a STEM major in 4-year colleges. The results indicate that NAEP explained slightly 
more variance than the SAT alone—about 10% compared to 9%. Interestingly, when GPA was 
added to the models, the variance explained did not change for either the NAEP or SAT 
models—both remained at 10% and 9%, respectively (Models 3 and 4). 


Table 14. Comparing NAEP and SAT mathematics scores in predicting the choice of a STEM 


major 
Variable Yoyo (=) ik WY CoXe (=) fd WV CoXe (=) 3 WY Coxe (=) ir.) 
0.026 0.028 
NAEP (0.002)*** (0.002)*** 
-0.150 0.016 
GPA (0.094) (0.078) 
0.007 0.007 
SAT (0.001)*** (0.001)*** 
Cutoff 1: Not 
choosing a STEM 5.536 4.615 5.348 4.640 
major (0.339)*** (0.306)*** (0.364)*** (0.339)*** 
Pseudo R? 0.103 0.087 0.104 0.087 
N 10,570 10,570 10,570 10,570 


***o < 001, **p<.01, *p<.05. 
SOURCE: U.S. Department of Education, National Center for Education Statistics, High School Longitudinal Study of 
2009 (HSLS:09). 
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Summary and Discussion 


There is an increased focus on ensuring that students are prepared for college when they 
graduate from high school. As the assessment “gold standard” of measuring what students 
know and can do, NAEP can potentially inform educators and policymakers on whether the 
nation’s schools have adequately prepared students for college by the time they reach 12th 
grade. There have been several studies that examined the relationship between NAEP 
achievement and college admission at the national level and postsecondary entry at the state 
level (Ogut, Bohrnstedt, Broer, 2019; Fields, 2014). However, this is the first study examining 
the relationship between NAEP achievement and postsecondary enrollment and other college 
outcomes using nationally representative data. The results showed that NAEP explained a 
considerable amount of variation in postsecondary outcomes, with about 28% of the variance 
explained in overall postsecondary enrollment and about 26% of the variance explained in the 
selectivity of the colleges enrolled in. However, the variance explained decreased considerably 
in predicting who chooses a STEM major, remedial coursetaking, and persistence—13%, 10%, 
and 6%, respectively. 


When translated into probabilities of enrollment, the results showed that the NAEP Basic level 
corresponded to a 33% probability of entry into any 4-year college but a 21% probability of 
entry into a moderately or highly selective 4-year college and a 6% probability of entry into a 
4-year not classified or inclusive college. A student at the NAEP Proficient level had a 64% 
probability of entry into any 4-year college, but an 18% probability of enrolling in a highly 
selective 4-year college, a 33% probability of enrolling in a moderately selective 4-year college, 
and an 8% probability of enrolling in a 4-year inclusive college. Although NAEP does not have a 
below NAEP Basic cut score, the average NAEP achievement of students who performed below 
the NAEP Basic cut score was used to compute the probabilities of entry into college. The 
results show that an average student who scored below NAEP Basic had a 19% probability of 
entry into any 4-year postsecondary education institution, a 2% probability of entry into a 
highly selective college, an 8% probability of enrollment in a moderately selective 4-year 
college, and an 8% probability of enrolling in a 4-year not classified or inclusive college. 


In addition to enrolling in postsecondary education, it is also important to examine whether a 
student is required to take remedial courses upon enrolling. Remedial coursetaking information 
was provided by the students themselves and therefore is prone to errors. Perhaps, for this 
reason, NAEP was not able to predict remedial coursetaking as well as it was able to predict 
entry into a postsecondary institution. In terms of probabilities, the NAEP Basic level was 
associated with a 49% probability of enrollment in a 2- or 4-year college without remedial 
coursetaking, whereas the NAEP Proficient and NAEP Advanced levels were associated with 77% 
and 94% probabilities of enrollment without remediation, respectively. 
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The association between grade 12 NAEP mathematics achievement and choosing a STEM major 
in 4-year colleges (R* = 0.171) was lower than that of NAEP’s relationship with attending college 
without remediation (R* = 0.234) and overall postsecondary enrollment (R* = 0.282). When 
transformed into probabilities, a student at the NAEP Basic achievement level had an 8% 
probability of choosing a STEM major, compared with 28% for students at the NAEP Proficient 
level and 52% for students at the NAEP Advanced level. 


The relationship of the NAEP achievement levels and persistence in college had the lowest 
variance explained, at 6.4%. The results showed that scoring at the NAEP Basic level was 
associated with a 72% probability of persistence, whereas the probability increased to 84% at 
the NAEP Proficient level and 92% at the NAEP Advanced level. Perhaps most surprising, of 
those who scored below NAEP Basic, there was a 63% probability of having completed or still 
being enrolled in college in 2016. The fact that over 50% of the students performing below 
NAEP Basic were enrolled in any college and that by 2016, 63% of them were still enrolled or 
had completed their studies is noteworthy. 


Analyses also examined the relationship between the Governing Board’s college preparedness 
indicator cut point and the various outcomes examined in this study. In all cases, the 
probabilities generated were slightly lower than those associated with the NAEP Proficient 
level. So, for example, the probability of enrollment in any college, given one was at the NAEP 
Proficient level, was found to be 64%, compared to 53% for someone at the NAEP preparedness 
cut point. As another example, the probability of enrollment in college without remediation for 
someone at the NAEP Proficient level was 77%, compared to a probability of 68% for someone 
at the Governing Board’s preparedness cut point. Finally, the probability of majoring in STEM at 
the NAEP Proficient level was found to be 28%, whereas the probability at the preparedness cut 
point was 7 percentage points lower—21 percent. These results provide the first external 
validity evidence for NAEP’s college readiness indicator from nationally representative data. 


Taken together, the analyses using the NAEP achievement levels provided substantial evidence 
of the predictive validity of NAEP for several important college-related outcomes: college 
attendance, the selectivity of the college attended, whether college is attended without the 
need for remedial coursetaking, persisting in college once there, and choosing a STEM major. 
One of the unexpected but important study findings was learning that the probability of 
choosing a STEM major was 52% for those at the NAEP Advanced level on the grade 12 NAEP 
mathematics assessment. This suggests that the interest and ability in majoring in STEM are 
high for those who score at this level, but the overall percentage of students who are in the 
NAEP Advanced category remains low (4%). 


51 


Research shows that postsecondary enrollment depends on several factors, including academic 
skills, noncognitive skills, and the availability of information about college systems (Roderic, 
Nagaoka, & Coca, 2009). Among the factors associated with college entry are academic skills 
measured by GPA or college entrance exams, students’ desire to go to college, SES, the 
availability of financial aid, parental education, peer effects, and high school involvement 
(Cartledge, Baldwin, Persall, & Woolley, 2015; Hossler & Gallagher, 1987; Darolia & Koedel, 
2018; McDonough & Antonio, 1996; Roderick, Nagaoka, & Coca, 2009). NAEP is a measure of 
academic skills and, therefore, the results of this study only reflect the association between 
college enrollment and academic achievement, without taking into account any of the other 
potentially important factors. Given this, perhaps it should not be surprising that NAEP did not 
explain a large amount of the variation in postsecondary outcomes examined in the study. 
However, NAEP performed on a par with the SAT, a test specifically designed for that purpose. 


An examination of the relationships between NAEP and the various outcomes by subgroup 
showed differences, but not always as expected. Perhaps the most surprising one was that at a 
given NAEP achievement level, Black students had higher probabilities of postsecondary 
enrollment than White or Hispanic students. The study’s examination of the selectivity of 
colleges enrolled in revealed that both Black and White students had higher probabilities than 
Hispanic students of enrollment in more, rather than less, selective colleges. White and Black 
students also had higher probabilities than Hispanic students of choosing STEM majors. White 
and Black students also had higher probabilities of postsecondary enrollment without 
remediation than did Hispanic students. There were no apparent differences in persistence 
across different racial/ethnic groups, but female students appeared more likely than male 
students to persist. It should be noted that the actual percentages of Hispanic and Black 
students at the NAEP Proficient and NAEP Advanced achievement levels (which are associated 
with higher positive outcome probabilities, regardless of group membership) are lower than 
those of White students. 


The higher probabilities of postsecondary enrollment for Black students could be due to 
affirmative action or to the efforts of college administration offices to ensure a diverse student 
population. However, one would have expected affirmative action to have a similar effect on 
Hispanic students’ enrollment probabilities at a given performance level, but they actually had 
lower probabilities of enrolling in colleges with higher selectivity. Studies show that financial aid 
is a significant predictor of the gap in enrollment between 2- and 4-year colleges for Hispanic 
students (O’Connor, Hammack, & Scott, 2010). Higher percentages of Hispanic students at 
2-year colleges were also observed in HSLS:09. The higher likelihood of admission of Black 
students to inclusive and moderately selective colleges also points to the potential interaction 
between academic achievement and other factors, such as SES. 


52 


The study also found that at each achievement level, female students had higher probabilities 
of postsecondary enrollment, enrolling in colleges with higher selectivity, enrolling without the 
need for remedial coursetaking, and persisting in college than male students. However, male 
students had higher probabilities than females of choosing STEM majors in 4-year colleges. 
Historically, males have had a higher rate of college attendance than females. For example, in 
1970, 32% of 18- to 24-year-old males were enrolled in college, compared to 20% of females of 
the same age. By 2015, a higher percentage of females than males were enrolled—the 
comparable percentages were 38 and 43 percent.? That is, over a 45-year period, college 
attendance for males increased by 18%, but for females the increase was much larger—113%. 


The fact that this study identified female students as having higher probabilities of enrollment, 
enrollment without remediation, enrolling in highly selective colleges, and persistence may be 
due to several factors. First, the results observed might reflect the use of gender as a factor in 
admission decisions. Second, the results that show males as being more likely to attend 
selective colleges are outdated and may not reflect female students’ increasingly high 
educational expectations, thereby resulting in more female students applying, and being 
admitted, to college in general and more selective colleges in particular. Finally, there might 
also be differences between male and female students in noncognitive factors associated with 
better outcomes. 


In terms of SES, students with higher SES had a higher probability of enrollment, enrolling in 
highly selective colleges, and enrolling without remedial courses than those with lower SES. 
These results are in line with the literature that shows that lower SES students are less likely to 
attend postsecondary colleges than are higher SES students (Long, Conger, & Latarola, 2012). 
However, there were no differences between students with lower SES and students with higher 
SES in choosing STEM majors. 


The results from this study showing different probabilities for postsecondary outcomes by 
subgroup do not necessarily suggest that students from different backgrounds are favored or 
being discriminated against. The results are mostly in line with the literature and reinforce the 
idea that academic skills, as reflected by NAEP achievement, are only one of the factors 
affecting the probability of postsecondary entry; other factors include socioeconomic status, 
financial aid, and students’ educational aspirations (Cartledge, Baldwin, Persall, & Woolley, 
2015; Darolia & Koedel, 2018; Roderick, Nagaoka, & Coca, 2009). The differences in any of 
these factors across student groups could lead to varying probabilities of college enrollment. 


The current study also compared the performance of the grade 12 NAEP mathematics 
assessment against the SAT mathematics assessment in predicting college enrollment. The 


° See https://nces.ed. gov/programs/digest/d16/tables/dt16_302.60.asp. 
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results showed that they performed similarly, explaining about 20% of the variance in 
postsecondary enrollment and about 30% of the variance in the selectivity of the college 
enrolled in. The NAEP and SAT mathematics assessments also performed similarly in terms of 
the prediction of remedial coursetaking, postsecondary persistence, and choosing a STEM 
major, explaining, on average, 12%, 5%, and 9.5% of the three variances, respectively. These 
results suggest that the NAEP grade 12 Mathematics Assessment did as well in explaining the 
variance in these outcomes as the SAT mathematics assessment did when both were looked at 
in combination with overall high school GPA. These findings provide further validation of NAEP 
as a measure of college preparedness. 


Taken together, the findings from the study provide strong evidence of the predictive validity of 
NAEP for a set of important outcomes related to college enrollment and attendance. This 
concerns not only the predictive validity of NAEP in general but also the NAEP achievement 
levels as well as the mathematics college preparedness indicator. The results of this study can 
further be used to identify alternative cut points on the NAEP scale (such as a 50%, 66%, or 75% 
chance of a positive outcome). 


It should be noted that the current results are nationally representative but are based on only 
two (enrollment and remedial coursetaking but not first-year college GPA of 2.7 or higher) of 
the three criteria that the Governing Board uses in defining college preparedness (Fields, 2014). 
Missing is GPA at the end of the college freshman year. Nevertheless, this study offers the best 
evidence yet of the validity of grade 12 NAEP mathematics achievement as a preparedness 
indicator. Another limitation of the study is that remedial coursetaking was self-reported. These 
limitations will be addressed in an upcoming follow-on study that incorporates HSLS:09’s 
recently released college transcript data, which includes actual instead of reported remedial 
coursetaking and first-year GPA. Indeed, the transcript data, which will include GPA through the 
completion of college, will provide an even better test of how well the Governing Board’s 
mathematics preparedness indicator predicts overall college academic performance. 
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Appendix A 


Figure A-1 below shows the probabilities of postsecondary enrollment under a scenario where 
students are assumed to have been able to enroll in a lower postsecondary education level 
than that for which they are qualified. The results show that a student at the NAEP Advanced 
level has a 97% probability of enrolling in a 2-year or less college, a student at the NAEP 
Proficient level has a 90% probability, a student at the NAEP Basic level has a 71% probability, 
and a student scoring below NAEP Basic has a 54% probability. 


Figure A-1. Cumulative probabilities of postsecondary enrollment 
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SOURCE: U.S. Department of Education, National Center for Education Statistics, 
High School Longitudinal Study of 2009 (HSLS:09). 


Figure A-2 shows the cumulative probabilities of enrolling in colleges with different degrees of 
selectivity. Not surprisingly, students at the NAEP Advanced level have the highest cumulative 

probability of enrollment in a moderately selective college (82%), whereas a below NAEP Basic 
student has the lowest probability (10%). 
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Figure A-2. Cumulative probabilities of enrolling in colleges with different degrees 
of selectivity, by NAEP achievement level 
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SOURCE: U.S. Department of Education, National Center for Education Statistics, 
High School Longitudinal Study of 2009 (HSLS:09). 


Table A-1. Comparison of analytic sample with NAEP operational sample 


Variable Analytic Sample NAEP Sample 
Percentages 
White 54 58 
Black 13 14 
Hispanic 21 20 
Asian 4 6 
Other race 8 3 
Male 50 51 
Scores 
NAEP score 150.62 153 


#Rounds to zero. 


SOURCE: U.S. Department of Education, National Center for Education Statistics, High School Longitudinal Study of 


2009 (HSLS:09). 


A-2 


Table A-2. Comparison of SAT analytic sample with main analytic sample 


SAT analytic sample Main analytic sample 

(N = 10,660) (N = 14,000) 
Group Mean SD Mean SD 
Percentages 
White 57% 49% 54% 50% 
Black 14% 34% 13% 33% 
Hispanic 17% 37% 21% 41% 
Asian 4% 21% 4% 20% 
Other race 8% 27% 8% 28% 
Male 47% 50% 50% 50% 
Parent college AA% 50% 37% 18% 
graduate 
eee higher than 57% 19% 51% 50% 
Scores 
Algebra, grade 11 53.35 9.61 50.91 9.97 
Algebra, grade 9 53.02 9.44 50.82 9.83 
GPA 2.97 0.66 2.70 0.80 


SOURCE: U.S. Department of Education, National Center for Education Statistics, High School Longitudinal Study of 
2009 (HSLS:09). 


Table A-3. Correlations among GPA, SAT math, and NAEP scale scores 


NAEP 
NAEP scale NAEP scale TAY ot Yor] (=) NAEP scale CYor=] (=) 


score score score score score 
(Enrollment) (Selectivity) (Persistence) (Remediation) (STEM) 


NAEP scale 

score 

(enrollment) 1.00 

NAEP scale 

score 

(selectivity) 1.00 1.00 

NAEP scale 

score 

(persistence) 0.79 0.79 1.00 

NAEP scale 

score 

(Remediation) 1.00 1.00 0.79 1.00 

NAEP scale 

score (STEM) 0.81 0.81 0.77 0.81 1.00 

GPA 0.67 0.66 0.62 0.67 0.67 1.00 
SAT math 0.74 0.74 0.72 0.74 0.73 0.58 1.00 


SOURCE: U.S. Department of Education, National Center for Education Statistics, High School Longitudinal Study of 
2009 (HSLS:09). 
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